Kullback's inequality について

Words near each other

Dictionary Lists

mini英和辞書

翻訳と辞書　辞書検索 [ 開発暫定版 ]

スポンサードリンク

Kullback's inequality ：ウィキペディア英語版

Kullback's inequality
In information theory and statistics, Kullback's inequality is a lower bound on the Kullback–Leibler divergence expressed in terms of the large deviations rate function. If ''P'' and ''Q'' are probability distributions on the real line, such that ''P'' is absolutely continuous with respect to ''Q'', i.e. ''P''<<''Q'', and whose first moments exist, then
:

D_(P\|Q) \ge \Psi_Q^*(\mu'_1(P)),

where

\Psi_Q^*

is the rate function, i.e. the convex conjugate of the cumulant-generating function, of

Q

, and

\mu'_1(P)

is the first moment of

P.

The Cramér–Rao bound is a corollary of this result.
==Proof==
Let ''P'' and ''Q'' be probability distributions (measures) on the real line, whose first moments exist, and such that ''P''<<''Q''. Consider the natural exponential family of ''Q'' given by
:

Q_\theta(A) = \fracQ(dx)}   = \frac \int_A e^Q(dx)

for every measurable set ''A'', where

M_Q

is the moment-generating function of ''Q''. (Note that ''Q''₀=''Q''.) Then
:

D_(P\|Q) = D_(P\|Q_\theta)   + \int_\left(\log\frac\right)\mathrm dP.

D_(P\|Q_\theta) \ge 0

so that
:

D_(P\|Q) \ge   \int_\left(\log\frac\right)\mathrm dP = \int_\left(\log\frac\right) P(dx)

Simplifying the right side, we have, for every real θ where

M_Q(\theta) < \infty:

D_(P\|Q) \ge \mu'_1(P) \theta - \Psi_Q(\theta),

where

\mu'_1(P)

is the first moment, or mean, of ''P'', and

\Psi_Q = \log M_Q

is called the cumulant-generating function. Taking the supremum completes the process of convex conjugation and yields the rate function:
:

D_(P\|Q) \ge \sup_\theta \left\   = \Psi_Q^*(\mu'_1(P)).

翻訳と辞書 : 翻訳のためのインターネットリソース